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As conventional structured light with temporal and spatial encoded pattern is used to retrieve 3D 
shape, its measurement accuracy is inevitably affected by the broken and dislocated fringes or grids 
appearing in shaded, prominent or edge areas. In this study, a non-encoding fringe strategy is presented 
for large field 3D shape measurement under infrared light illumination. By utilizing the images 
recorded from several different orientations, the break and dislocation problems can be effectively 
solved by an algorithm of automatic identification and matching for out-of-plane height curves based 
on Euclid distance and quaternion. Meanwhile, the field of view can also be enlarged by prolonging the 
fringes in the edge areas. The examples of retrieving 3D shapes of the statues demonstrate the 
validation of the proposed approach. 

Crown Copyright © 2012 Published by Elsevier Ltd. All rights reserved. 


1. Introduction 

Various structured light methods [1], mostly with time¬ 
multiplexing and spatial codification strategies, have been widely 
applied to retrieve 3D surface shapes. Among the temporal coding 
schemes, the binary encoding technique is to be simple and 
convenient as only two illumination levels are used [1,2]. Because 
a large number of patterns need to be projected on the surface to 
form the image series with time sequence, however, the temporal 
encoding patterns are more suitable for static measurement. To 
increase acquisition speed for applications to moving object 
measurement, a variety of spatial encoding structured light 
schemes have been developed. Phase retrieval techniques has 
been applied in [3-6] reconstructed 3D surface by using various 
phase unwrapping algorithms; color in Refs.[7,8] proposed 
encoded fringe patterns and applied the color information of 
neighboring stripes to retrieve 3D shape. Nevertheless, a big 
problem exists in ah of those temporal and spatial encoding 
structured patterns that the shaded, prominent and edge areas on 
the 3D surfaces lead to the acquired fringes or grids to be broken 
or dislocated so that there are inevitable errors and even losses of 
the measurement data [1,9]. In addition, these codification 
methods are usually affected by noises in various light environ¬ 
ments. For the problem resulting from the shadow and tilted 
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areas on surface, Kowarschik et al. [9] proposed a scheme of 
rotating the object by more than three different angles so that the 
whole surface could be illuminated with the structured light to 
complete the lost fringe information in the images with local 
shadows by those acquired from other orientations. This scheme 
provided a valuable way to avoid data loss or mistakes so as to 
reconstruct whole 3D surface from many overlapping views of 
different directions, even though the phase unwrapping proces¬ 
sing was complicated. 

In this paper, a novel approach of non-encoding structured 
light is proposed in capturing images from different directions to 
reconstruct large field 3D surface. Unlike the sinusoidal patterns 
used in phase retrieval methods, in the design of the projecting 
pattern of this study, only two illumination levels, 0 and 1, are 
used periodically, without any temporal or spatial encoding. As a 
pattern of binary encoding, it has a variety of advantages such as 
relatively high spatial resolution, easy segmentation of the 
viewed patterns and robustness against noise [1,2]. To obtain 
the height information of a surface as large as possible, the object 
is illuminated three times from different directions to ensure the 
fringes with shadow and tilted regions in one of the pictures to be 
smoothly distributed in the other one. Because the projected 
pattern is very simple in structure without any coding, the 
segmentations of those fringes become much easier and more 
accurate, which thus makes the out-of-plane height curves 
resulting from different views be easily identifiable. In the 
combination of the images from neighboring shooting, we pro¬ 
pose a correlation algorithm that can correctly match the 
out-of-plane height curves together with the solution of rotation 
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angles during pattern projections. Moreover, the binary level of 
fringe intensity used in this projection makes it possible for the 
pattern illumination using either visible light or infrared source, 
the latter condition of measurement will be important for night 
vision applications or in the environments with light noises [10]. 


2. Material and methods 

Fig. 1(a) shows the optical system for surface height measure¬ 
ment. An infrared projector is used to illuminate the specimen 
with the un-encoded structured pattern in an inclined direction, 
and an infrared digital camera (computar M1214-MP2, resolution: 


2592x1944) is mounted to capture the images in the normal 
direction to the reference plane. Fig. 1(b) presents the structure of 
the infrared projector, from left to right: a 3 W infrared light 
source; a 12.1 in rear Fresnel lens with 220 mm focus; a grating 
with 0.7 line/mm; a 12.1 in front Fresnel lens with 330 mm focus; 
and a projecting lens with focus length of 330 mm. In this system, 
the object under test is placed on a rotating plane, where the 
rotating axis z is perpendicular to the projection direction. 

A nose statue is used as a typical model of shape measurement 
where the wings of the nose commonly produce image shadows 
during the front projection on the surface. The fringe patterns 
captured from three different orientations are shown in Fig. 2(a), 
(b) and (c), respectively, denoted as the view directions a, /? and y 



Fig. 1. Schematic illustration of the measurement system with infrared structured light: (a) the optical geometry of projection for triangulation relationship: (b) the 
infrared projector structure, from left to right: a 3 W infrared light source; a 12.1 in rear Fresnel lens with 220 mm focus; a grating with 0.7 line/mm; a 12.1 in front Fresnel 
lens with 330 mm focus; and a projecting lens with focus length of 330 mm. 



Fig. 2. Pattern pictures captured from the directions of (a) view a; (b) view/1; (c) view y; and the corresponding segment results: (d), (e) and (f), where the circles indicate 
the areas with dislocated fringes due to illumination shadows. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of 
this article.) 
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as the object rotated around axis z. The corresponding segmented 
fringes are shown in Fig. 2(d), (e) and (f), where in 
Fig. 2(e) corresponding to the image of Fig. 2(b) resulting from 
normal capturing, the yellow circles point out several broken and 
dislocated fringes caused by the shaded and prominent parts of 
the test object, while the fringes in the corresponding regions of 
Fig. 2(d) and (f), on the other hand, are smoothly connected. By 
using the information of those easily segmented fringes with 
respect to neighboring views, a process of five steps with auto¬ 
matic out-of-plane height curves identification and matching is 
described in the following to reconstruct the wide field 3D surface 
shape. 

1. Detecting the fragmented and dislocated fringes: For any given 
vertical line, no more than one consecutive part in the 
fragmented fringe goes through it; thus, only one continuous 
part in the fragmented fringe will persist. The method devel¬ 
oped in Step 4 can detect the dislocated fringes. 

2. Using conventional triangulation relationship to calculate the 
out-of-plane height curves of the corresponding fringes: 
The out-of-plane height data are needed to obtain rotation 
matrix and final reconstruction. For convenience, each out-of- 
plane height curve is marked by a unique serial number; for 
instance, the first curves in the views of a, /? and y are 
represented as the curves ai, /?i and ^respectively. Because 
the acquired infrared fringes are neither spatial nor temporal 
encoded, we just need to identify which two curves in the 
neighbor shooting angles are matched to each other in an 
alternative way. Based on the truth that there exists an 
overlapping segment in the out-of-plane height curves among 
different views, the segment is used to recognize the corre¬ 
sponding matched curves. Without loss of generality, we 
choose k points (for instance, k= 300) in the overlapped 
segment for matching calculation. 

3. Calculating the rotation matrix between two curves from 
neighboring images. The first curve is represented as 
Ci = [Xu • • -x lk ; y n • • -y lk ; 1 • • • 1], and the second curve is 
C 2 = [*2i • • • X 2k ; y 2 \ • • -y 2k ; 1 ■ • • 1], where k is the number of 
the point in each part of curve. The centroid of the ith curve is 

(XiJi) = ((l/k)(E?=iXi).(l/k)(Ef=iyij). where i= 1, 2. By 
adding the coordinate difference(Ax,Ay) = (xi -x 2 ,yi -y 2 )to the 
second curve, the two curves can be translated to the same 
centroid, and the second curve after translation is denoted as 
C t = [x t i • • x tk ; y tl • • y tk ; 1 • • • 1]. In our study, the quaternion 
in [11] is applied to estimate the rotation matrix among 
different views; thus, the orthogonal rotation matrix R is 
obtained by R=M(M T M )~ 1/2 , whereM = Ci Cj, ensuring that 
the curve C t has no deformation in further rotation procedure. 

4. Identifying and matching the out-of-plane height curves: 
The matched curve is explored at neighbor shooting angle in 
certain ranges by traversal of the same length curves and 
solving the rotation matrix. For those patterns shown in Fig. 2, 
the curves y 2 -y& in Fig. 2(f) are chosen as the candidates. 
Take curve y 2 and /? 4 for instance; after translating and 
rotating the two curves, the total Euclid distance of all 
corresponding points in two possible overlapped curves is 

d= E \j (*i 1 -Xti) 2 + (yij-y t i) 2 • By calculating d between all 

curves with k points in length in the curves 72-76 and the 
overlapped segment in the curve /? 4 , there must exist a part 
of curve in the view 7 for which d reaches the minimum value. 
Generally, the overlaps on the dislocated out-of-plane height 
curves do not really exist on the test object; thus the value 
of d between the curves generated by dislocated fringes 
and overlapped segment is always very large. Therefore, those 



x/mm 

Fig. 3. A curve identification and matching result. The colorful lines are out-of- 
plane curves from three projected directions: the green one is a 4 , the red one is /? 4 
and the blue one is y 4 . The colorful lines are transferred by the lines in Fig. 2 by 
using triangulation relationship. (For interpretation of the references to color in 
this figure legend, the reader is referred to the web version of this article.) 

dislocated fringes can be easily detected and then eliminated 
in further matching computations. In this example, the curve 
7 4 is proved to match with curve /? 4 , and the displacements of 
the centroid coordinate of curve before and after rotation 
are the translation parts of the rotation and translation matrix. 
Fig. 3 presents a typical matching result based on the curve 
identification and matching mentioned above. The pink line, 
the green line and the red line in Fig. 3 show the acceptable 
matching result of three typical out-of-plane height curves a 4 , 
and 7 4 , which correspond to the colorful lines in Fig. 2(d), 
(e) and (f), respectively. 

5. Matching all out-of-plane height curves in different views into 
a unique coordinate system: The rotation and translation 
matrix between view a and view /? is denoted as R\ 2 , and the 
matrix between view and view 7 is denoted as R 23 . Because 
there are overlaps between view a and view 7, all curves in the 
view a can match with all curves in the view 7 on multiplying 
R=R] 2 R 23 . In this way, 3D reconstruction of the object surface 
is realized. 


3. Results 

To prove the efficiency of the proposed method, the 3D 
reconstruction results of two statues are presented in 
Figs. 4 and 5. There exist overlaps between view a and view 7, 
so only two captured directions are presented in Figs. 4(a) and 
5(b). Fig. 4(a) shows the 3D height points of the nose statue 
viewed in three different orientations, where the red area is 
obtained from the view a and the blue area is acquired from 
the view 7. The included angle between view a and view /3 is 53°, 
and the intersection angle between view and view 7 is 60°. 
Fig. 4(b) gives the 3D surface reconstruction of the nose corre¬ 
sponding to Fig. 4(a). For further demonstration, similar proces¬ 
sing is carried out for a face statue as shown in Fig. 5, where 

(a) gives the fringe patterns captured from three different views, 

(b) shows the 3D height points of the identified curves with 
matching from neighboring images of shooting, and (c) presents 
the reconstructed results of the face surface. The included angle 
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Fig. 4. (a) 3D points of a nose statue resulting from identifying and matching of height curves from different orientations, with the red part corresponding to the view a 
and the blue part to the view y and (b) reconstructed 3D shapes shown in different views with the view angle of 290°. (For interpretation of the references to color in this 
figure legend, the reader is referred to the web version of this article.) 


a 



Fig. 5. (a) Fringe images of a face statue shot from three directions; (b) 3D points of the face statue resulting from identification and matching processing, with the red part 
corresponding to the view a and the blue part to the view y; and (c) 3D reconstruction result of the face statue with a view angle of 260°. (For interpretation of the 
references to color in this figure legend, the reader is referred to the web version of this article.) 
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between view a and view /3 is 33°, and the intersection angle 
between view ft and view y is 42°. 

In this experiment, the distance between the reference plane 
and infrared camera is 1.2 m, and the distance between infrared 
camera and infrared projector is 30 cm. In addition, the angle 
between the projecting and the receiving directions is 26°. With 
this system layout, the accuracy is around 0.5 mm in height 
measurement. In fact, the visual angle can be enlarged to 290°. 
Because only the distortion of central lines of fringes are recorded 
and used for out-of-plane height calculation, the accuracy of this 
method is relatively lower than those of the techniques in [3-6], 
but the non-encoding structured light approach can reduce 
information loss caused by shadow and prominent parts, thus 
making the measurement less sensitive to noise. Full-field mea¬ 
surement would be retrieved by adding the number of projected 
directions to four or more. 

4. Conclusion 

In conclusion, the non-encoding of the structured light allows 
the projected fringes easily segmented to produce the out-of- 
plane height curves and correctly matched for those captured 
from different views to eliminate the errors of measurement in 
the shaded, prominent areas caused by projection. Consequently, 
the range of the measured surface can be enlarged to 290° by 
extending these non-encoding fringes to the edge areas under 
multiple views. Moreover, based on those infrared structured 
fringes, the presented approach is less sensitive to light 


environments and could provide night viewing measurement. 
Further studies will focus on dynamic measurement of full-field 
3D shape using multiple cameras from different views. 
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